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The  words  “integrated” 
and  “integration”  have 
appeared  for  many  years 
in  their  association  with 
defense  acquisition.  Al¬ 
though  integration  has  been  discussed 
and  written  about  extensively,  making  it 
happen  has  not  been  successful  in  such  a 
non-integrated  environment  of  multiple 
Services  and  agencies,  multiple  contrac¬ 
tors,  multiple  responsible  test  organiza¬ 
tions,  and  multiple  customers.  In  1983, 

Congress  looked  towards  test  and  eval¬ 
uation  (T&E)  officials  to  take  a  giant 
step  toward  integration  in  defense  ac¬ 
quisition  by  incorporating  responsibility  for  operational 
T&E  with  designated  Service  and  agency  Operational 
Test  Agencies  (OTAs),  and  the  creation  of  the 
Director  of  Operational  Test  and  Evaluation 
(DOT&E).  This  move  created  an  integrated  chain  of 
testing,  evaluating,  and  reporting  at  the  completion  of 
major  defense  acquisition  programs  (MDAPS)  to  the 
two  customers  of  defense  acquisition:  1)  the  warfighter 
who  uses  the  equipment,  and  2)  Congress,  representing 
the  U.S.  taxpayer  who  pays  for  the  equipment.  While 
this  integration  of  responsibility  and  authority  for 
operational  T&E  (OT&E)  has  been  very  effective,  a 
drawback  is  that  OT&E  by  nature  requires  test  items 
that  are  near  the  completion  of  development  so  that 
they  can  be  operated  by  their  ultimate  users  in  an 
operationally  representative  environment.  Therefore, 
the  majority  of  OT&E  must  be  accomplished  near  the 
end  of  the  development  cycle.  Here  within  lies  the 
problem;  too  much  “stuff  happens”  during  the  earlier, 
significantly  non-integrated,  developmental  part  of  the 
acquisition  process  that  the  two  key  customers  above 
have  no  knowledge  of  or  influence  over.  The  result  is 
that  OT&E  becomes  “discovery”  of  problems  that 
could/should  have  been  dealt  with  earlier  in  the  process 
when  they  would  have  been  much  less  significant. 
While  integrated  test  and  evaluation  has  been  a  recent 
focus  with  several  policy  statements  issued,  there  was 
no  pre-IOT&E  stakeholder  in  place  to  ensure  early 
integrated  testing  of  systems.  In  2009,  Congress  once 
again  turned  to  T&E  to  integrate  this  part  of  the 
process  as  well  with  the  creation  of  the  office  of  the 


Director,  Developmental  Test  and  Eval¬ 
uation  (DDT&E).  Thus  the  responsi¬ 
bility  of  the  DDT&E  is  to  assure  that 
developmental  test  and  evaluation 
(DT&E)  is  effective,  visible,  and  inte¬ 
grated  with  OT&E  to  form  a  knowledge 
continuum  throughout  the  entire  devel¬ 
opment  and  acquisition  process. 

Integrated  Testing  is  defined  by  OSD 
Memo,  “Definition  of  Integrated  Test¬ 
ing,”  dated  25  April  2008,  as  follows: 
“the  collaborative  planning  and  collabo¬ 
rative  execution  of  test  phases  and  events 
to  provide  shared  data  in  support  of 
independent  analysis,  evaluation,  and 
reporting  by  all  stakeholders,  particularly  the  develop¬ 
mental  (both  contractor  and  government)  and  opera¬ 
tional  test  and  evaluation  communities.” 

From  my  perspective,  the  word  “integrated”  has  four 
key  meanings  associated  with  defense  acquisition. 
First,  integrated  T&E  must  be  an  integral  part  of 
development  and  acquisition.  Effective  and  efficient 
development  and  acquisition  absolutely  requires  exten¬ 
sive,  timely,  accurate,  and  impartial  knowledge,  and 
that  is  the  product  of  good  T&E.  While  the  defense 
development  and  acquisition  process  has  two  custom¬ 
ers,  the  warfighter  and  the  U.S.  taxpayer,  T&E  has  a 
customer  list  that  also  includes  the  program  manager, 
the  contractors,  the  program  management  team,  and 
the  entire  development  team. 

The  second  meaning  of  integrated  T&E  is  that 
contractor  and  government  DT&E  must  be  planned 
and  conducted  in  a  manner  such  that  there  is  no 
duplication  of  effort,  facilities,  personnel,  or  other 
resources.  Integrated  contractor  and  government  T&E 
must  also  include  the  open  sharing  of  test  data  in  order 
to  achieve  efficiencies.  Integrated  contractor  and 
government  DT&E  also  describe  a  smooth  and 
efficient  transition  from  very  early,  mostly  contractor 
conducted,  highly  technical  testing  of  components  and 
subsystems  to  the  often  more  government  conducted 
full  system  technical  testing.  Government  and  con¬ 
tractor  integrated  T&E  throughout  the  entire  devel¬ 
opment  will  assure  a  more  streamlined  and  cost 
effective  process  and  assure  that  the  knowledge  gained 
is  used  to  the  maximum  extent  possible  to  support 
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Figure  1.  The  Integrated  T&E  Continuum. 


timely  and  cost  effective  development  of  effective 
equipment. 

The  third  meaning  of  integrated  T&E  describes  the 
continuance  of  a  smooth  and  integrated  flow  of  T&E 
from  DT&E  with  and  into  OT&E.  Figure  1  shows  the 
resulting  continuum  of  T&E.  This  continuum  ranges 
from  pre-milestone  A  translation  of  user  requirements 
to  follow-on  T&E  (FOT&E)  and  product  sustain¬ 
ment  post-milestone  C.  The  engineering  and  manu¬ 
facturing  development  (EMD)  phase  of  the  system 
acquisition  life  cycle  is  a  recognizable  transition  period 
from  a  subsystem  engineering  effort  to  a  production- 
representative  system  evaluation  effort.  The  full 
spectrum  of  integrated  DT&E  and  OT&E  becomes 
most  evident  during  this  phase,  with  government 
DT&E  playing  an  increasingly  important  role.  The 
government  DT&E  role,  while  complimentary  to  both 
systems  engineering  and  operational  test  and  evalua¬ 
tion  efforts,  requires  an  entirely  different  set  of  skills 
and  resources  than  these  other  domains.  Specifically 
these  are  engineering  skills  and  resources  focused  on 
developing  systems  for  subsequent  operational  test  and 
employment.  The  integrated  T&E  continuum  allows 
for  efficiency  across  contractor  DT&E,  government 
DT&E,  and  government  OT&E.  As  shown  in  the 
blue  shaded  triangle,  systems  engineering,  when 
combined  with  (primarily)  contractor  test  capability, 
excels  at  realizing  system  specifications  in  component 
level  development.  Figure  1  depicts  how  this  effort 
continues  across  the  EMD  phase,  ultimately  resulting 
in  system-level  prototypes.  As  the  EMD  phase 
progresses,  the  government  test  community  starts  to 
work  with  the  contractor  test  community  to  gain 
insight  into  the  suitability  and  effectiveness  of  the 
engineering  design.  DOT&E  “owns”  the  assessment  of 


suitability  and  effectiveness  and  is  most  notably 
involved  in  pre-milestone  C  efforts  with  (early) 
operational  assessments  (OA).  A  T&E  continuum 
integrates  operational  assessment  with  engineering 
focused  verification  of  contractual  requirements,  an 
essential  step  to  assure  efficient  development  and 
acquisition  of  operationally  effective  and  suitable 
systems. 

DDT&E  provides  knowledge  to  support  engineer¬ 
ing  verification  of  contractual  requirements  and 
engineering  evaluation  of  military  weapon  systems. 
This  knowledge  supports  the  essential  transition  from 
how  we  expect  the  system  to  work  to  how  the  user 
needs  it  to  work  for  successful  employment.  Not  even 
the  most  robustly  engineered  set  of  requirements  can 
fully  capture  the  intent,  interactions,  or  dynamics  of 
the  operational  environment.  Therefore,  DDT&E 
provides  program  managers  with  an  understanding  of 
how  systems  will  perform  in  the  hands  of  the 
warfighter,  early  enough  to  influence  system  develop¬ 
ment.  DOT&E  provides  the  assessment  of  system 
effectiveness  and  suitability;  however  OTAs  are  not 
staffed  to  support  daily  interactions  with  the  product 
development  community.  In  addition,  while  OTAs  do 
well  at  replicating  the  user  environment,  they  are  not 
resourced  or  trained  to  isolate  engineering  parameters 
within  that  environment  and  provide  technical  feed¬ 
back  for  development.  An  integrated  T&E  continuum 
assures  that  both  happen  as  and  when  they  need  to  for 
maximum  efficiency  and  effectiveness  in  system 
development  and  acquisition. 

My  fourth  and  final  characteristic  of  the  word 
“integrated”  applies  to  my  responsibility  for  bringing 
together  and  assuring  adequacy  of  the  multitude  of 
capabilities  essential  to  support  good  T&E  for  defense 
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Figure  2.  DDT&E  Integrated  Responsibility. 


development  and  acquisition.  As  depicted  in  Figure  2, 
this  responsibility  ranges  from  well  tested  and  under¬ 
stood  maturing  technologies,  development  and  mainte¬ 
nance  of  a  professional  T&E  workforce,  a  robust  and 
efficiently  utilized  test  base  capability,  thorough  plan¬ 
ning  for  integrated  T&E  throughout  the  entire 
acquisition  program,  well  organized  and  responsible 
test  organizations,  to  effective  utilization  of  knowledge 
from  T&E  applied  to  key  acquisition  decisions.  As  part 
of  execution  of  this  responsibility,  my  organization  is 
working  with  the  Defense  Acquisition  University 
(DAU)  to  improve  the  T&E  training  and  certification 
courses  and  the  T&E  portion  of  other  related  curricula. 
Another  key  part  of  the  execution  of  my  responsibility  is 
to  integrate  and  develop  methodologies  and  best 
practices  for  T&E  of  emerging  technologies.  This 
includes  infrastructure  and  processes  to  test  hypersonic 
systems,  directed  energy  weapons,  non-lethal  weapons, 
next  generation  UAVs,  data  fusion,  anti-tamper,  cyber, 
and  complex  multi-node  mobile  networks.  DDT&E 
has  a  key  role  in  the  Secretary  of  Defense’s  (SECDEF’s) 
efficiency  initiatives,  streamlining  T&E  planning  and 
reporting  processes  and  documentation.  Closely  aligned 
with  this  SECDEF  initiative  is  our  initiative  to  assess 
the  cost  of  doing  T&E  business.  Such  an  assessment  will 
touch  all  of  our  areas  of  integrated  responsibility  and 
provide  insights  into  improvement  and  metrics  for 
continued  monitoring.  DDT&E  is  also  committed  to 
how  DT&E  can  be  highly  focused  to  enhance,  not 
delay,  rapid  acquisition.  Overall,  this  integration  of 
responsibility  for  policy,  people,  and  infrastructure  into  a 
single  organization  positions  DDT&E  to  contribute 
significantly  to  more  effective  and  more  responsive 
defense  development  and  acquisition. 


As  we  work  together  to  implement  and  improve 
integrated  T&E  across  the  continuum,  I  ask  for  your 
help  in  meeting  several  key  challenges  that  face  the 
T&E  community.  Three  of  the  more  pressing  chal¬ 
lenges  include  T&E  in  the  cyber  world,  achieving  the 
right  balance  of  T&E  within  Rapid  Fielding,  and 
achieving  greater  DoD  efficiencies  within  T&E.  We 
must  harness  the  intellectual  talent  of  our  skilled 
workforce  to  understand  and  develop  effective  ways  to 
test  and  assess  system  performance  and  assurance  in  the 
complicated  world  of  cyber  warfare.  Within  the 
initiative  to  rapidly  field  weapon  systems  to  the 
warfighter,  we  cannot  afford  the  proven  and  deliberate 
T&E  methodology  required  in  formal  acquisition — we 
must  find  effective  ways  to  quickly  test  and  assess 
capabilities  and  limitations  of  systems  as  they  are 
expedited  to  the  front  lines.  And  finally,  we  owe  it  to 
the  end  user  to  take  a  hard  look  at  our  processes, 
policies,  and  organizations  to  find  significant  efficiencies 
in  the  way  we  do  business.  These  challenges  must  be  met 
in  order  to  deliver  affordable  weapon  systems  that  work, 
and  I  ask  your  help  in  meeting  these  challenges. 

Truly,  integration  is  the  golden  key  to  successful 
(effective  and  efficient)  defense  development  and 
acquisition.  Because  the  T&E  community  and  processes 
reach  out  and  touch  many  key  elements  of  defense 
development  and  acquisition,  T&E  is  uniquely  posi¬ 
tioned  to  facilitate,  guide,  monitor,  assess,  and  report 
the  progress  and  effectiveness  of  this  integration. 
DDT&E  is  a  key  organization  within  USD(AT&L) 
assuring  that  integrated  T&E  is  conducted  to  signifi¬ 
cantly  improve  defense  development  and  acquisition. 
Without  a  doubt,  knowledge  is  the  power  to  make  it 
happen  and  T&E  is  the  conduit  for  that  knowledge.  □ 
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Edward  R.  Greer  was  sworn  in  as  the  Director  of 
Developmental  Test  and  Evaluation  (DDT&E)  on 
March  15,  2010.  He  serves  as  the  principal  advisor  on 
developmental  test  and  evaluation  (DT&E)  to  the 
Director  of  Defense  Research  and  Engineering  and  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology 
and  Logistics.  Mr.  Greer  is  responsible  for  developing  and 
revising  DT&E  policy  in  support  of  the  acquisition  of 
major  Department  of  Defense  (DoD)  weapon  systems. 
Other  significant  duties  include  reviewing  and  improving 
the  organization  and  capabilities  of  the  military  depart¬ 
ments  with  respect  to  DT&E  and  providing  advocacy, 
oversight,  and  guidance  to  elements  of  the  acquisition 
workforce  responsible  for  DT&E. 

Prior  to  this  political  appointment  and  since  2002,  Mr. 
Greer  served  as  the  Deputy  Assistant  Commander  for  Test 
and  Evaluation  (AIR  5.0A),  Naval  Air  Systems 
Command  and  Executive  Director,  Naval  Air  Wafare 
Center  Aircraft  Division  (NAWCAD),  Patuxent  River, 
MD.  As  the  senior  civilian  for  naval  aviation  T&E,  Mr. 
Greer  was  responsible  for  planning,  executing,  analyzing, 
and  reporting  of  all  naval  aviation  T&E  spanning  a 
workforce  of 6,600  and  an  operating  budget  of  almost  $1B. 
As  executive  director,  NAWCAD,  responsibilities  included 
ensuring  that  NAWCAD  technical,  business,  and financial 
objectives  were  met  across  a  workforce  of  14,400  and  a 
total  operating  budget  of  over  $4  billion. 

Mr.  Greer  joined  the  senior  executive  service  (SES)  in 


1998  as  director  of  the  Atlantic  Ranges  and  Facilities, 
NAWCAD,  responsible  for  all  facets  relating  to  the 
development,  maintenance,  and  operation  of  the  range 
and  test  facility  components  of  the  Navy’s  principal  air 
combat  systems  test  activity. 

From  1995  to  1998,  Mr.  Greer  served  as  principal 
deputy  program  manager  of  Airborne  Strategic  Command, 
Control,  Communications;  Program  Executive  Office  for 
Air,  Antisubmarine  Warfare,  Assault  &  Special  Mission 
Programs.  Mr.  Greer  was  responsible  for  all  aspects  of 
acquisition  including  systems  engineering,  logistics,  train¬ 
ing  systems  and  T&E.  From  1993  to  1995,  Mr.  Greer 
took  an  assignment  in  the  Pentagon  as  a  staff  specialist  in 
the  Office  of  Under  Secretary  of  Defense  for  Acquisition 
and  Technology,  Test,  Systems  Engineering  and  Evalu¬ 
ation;  Test  Facilities  and  Resources.  Prior  to  1993,  Mr. 
Greer  served  in  various  leadership  and  engineering 
positions  within  the  Naval  Air  Systems  Command  and 
was  the  Navy’s  representative  on  the  2007  Defense  Science 
Board  Task  Force  on  Developmental  Test  and  Evaluation. 

Mr.  Greer  is  a  past  president  of  the  Southern  Maryland 
Chapter  of  ITEA.  He  earned  his  bachelor  of  science  degree 
in  electrical  engineering  from  the  University  of  Maryland, 
College  Park  and  received  a  masters  of  science  degree  in 
management  from  the  Florida  Institute  of  Technology.  Mr. 
Greer  is  also  a  graduate  of  the  Defense  Systems 
Management  College  Program  Management  Course.  E- 
mail:  cdbrown.gm@gmail.  com. 
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